A framework to assess the realism of model structures using hydrological signatures

The use of flexible hydrological model structures for hypothesis testing requires an objective and diagnostic method to identify whether a rainfall-runoff model structure is suitable for a certain catchment. To determine if a model structure is realistic, i.e. if it captures the relevant runoff processes, both performance and consistency are important. We define performance as the ability of a model structure to mimic a specific part of the hydrological behaviour in a specific catchment. This can be assessed based on evaluation criteria, such as the goodness of fit of specific hydrological signatures obtained from hydrological data. Consistency is defined as the ability of a model structure to adequately reproduce several hydrological signatures simultaneously while using the same set of parameter values. In this paper we describe and demonstrate a new evaluation Framework for Assessing the Realism of Model structures (FARM). The evaluation framework tests for both performance and consistency using a principal component analysis on a range of evaluation criteria, all emphasizing different hydrological behaviour. The utility of this evaluation framework is demonstrated in a case study of two small headwater catchments (Maimai, New Zealand, and Wollefsbach, Luxembourg). Eight different hydrological signatures and eleven model structures have been used for this study. The results suggest that some model structures may reveal the same degree of performance for selected evaluation criteria while showing differences in consistency. The results also show that some model structures have a higher performance and consistency than others. The principal component analysis in combination with several hydrological signatures is shown to be useful to visualise the performance and consistency of a model structure for the study catchments. With this framework performance and consistency are evaluated to identify which model structure suits a catchment better compared to other model structures. Until now the framework has only been based on a qualitative analysis and not yet on a quantitative analysis.